{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MCMC Sampling\n",
    "\n",
    "The  `CmdStanModel` class method  `sample` invokes Stan's adaptive HMC-NUTS\n",
    "sampler which uses the Hamiltonian Monte Carlo (HMC) algorithm\n",
    "and its adaptive variant the no-U-turn sampler (NUTS) to produce a set of\n",
    "draws from the posterior distribution of the model parameters conditioned on the data.\n",
    "It returns a `CmdStanMCMC` object\n",
    "which provides properties to retrieve information about the sample, as well as methods\n",
    "to run CmdStan's summary and diagnostics tools.\n",
    "\n",
    "In order to evaluate the fit of the model to the data, it is necessary to run\n",
    "several Monte Carlo chains and compare the set of draws returned by each.\n",
    "By default, the `sample` command runs 4 sampler chains, i.e.,\n",
    "CmdStanPy invokes CmdStan 4 times.\n",
    "CmdStanPy uses Python's `subprocess` and `multiprocessing` libraries\n",
    "to run these chains in separate processes.\n",
    "This processing can be done in parallel, up to the number of\n",
    "processor cores available."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "\n",
    "\n",
    "CmdStanPy displays progress bars during sampling via use of package [tqdm](https://github.com/tqdm/tqdm).\n",
    "In order for these to display properly, you must have the \n",
    "[ipywidgets](https://ipywidgets.readthedocs.io/en/latest/index.html) package installed,\n",
    "and depending on your version of Jupyter or JupyterLab, you must enable it via command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!jupyter nbextension enable --py widgetsnbextension"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more information, see the the\n",
    "[installation instructions](https://ipywidgets.readthedocs.io/en/latest/user_install.html#), \n",
    "also [this tqdm GitHub issue](https://github.com/tqdm/tqdm/issues/394#issuecomment-384743637).\n",
    "\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fitting a model to data\n",
    "\n",
    "In this example we use the CmdStan example model\n",
    "[bernoulli.stan](https://github.com/stan-dev/cmdstanpy/blob/master/test/data/bernoulli.stan)\n",
    "and data file\n",
    "[bernoulli.data.json](https://github.com/stan-dev/cmdstanpy/blob/master/test/data/bernoulli.data.json>).\n",
    "\n",
    "We instantiate a `CmdStanModel` from the Stan program file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from cmdstanpy import CmdStanModel, cmdstan_path\n",
    "    \n",
    "bernoulli_dir = os.path.join(cmdstan_path(), 'examples', 'bernoulli')\n",
    "stan_file = os.path.join(bernoulli_dir, 'bernoulli.stan')\n",
    "data_file = os.path.join(bernoulli_dir, 'bernoulli.data.json')\n",
    "\n",
    "# instantiate, compile bernoulli model\n",
    "model = CmdStanModel(stan_file=stan_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, the model is compiled during instantiation.  The compiled executable is created in the same directory as the program file.  If the directory already contains an executable file with a newer timestamp, the model is not recompiled.\n",
    "\n",
    "We run the sampler on the data using all default settings:  4 chains, each of which runs 1000 warmup and sampling iterations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run CmdStan's sample method, returns object `CmdStanMCMC`\n",
    "fit = model.sample(data=data_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `sample` method returns a `CmdStanMCMC` object, which contains:\n",
    "- metadata\n",
    "- draws\n",
    "- HMC tuning parameters `metric`, `step_size`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('sampler diagnostic variables:\\n{}'.format(fit.metadata.method_vars_cols.keys()))\n",
    "print('stan model variables:\\n{}'.format(fit.metadata.stan_vars_cols.keys()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fit.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The sampling data from the fit can be accessed either as a `numpy` array or a pandas `DataFrame`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(fit.draws().shape)\n",
    "fit.draws_pd().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Additionally, if `xarray` is installed, this data can be accessed another way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fit.draws_xr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ``fit`` object records the command, the return code,\n",
    "and the paths to the sampler output csv and console files.\n",
    "The string representation of this object displays the CmdStan commands and\n",
    "the location of the output files.\n",
    "\n",
    "Output filenames are composed of the model name, a timestamp\n",
    "in the form YYYYMMDDhhmm and the chain id, plus the corresponding\n",
    "filetype suffix, either '.csv' for the CmdStan output or '.txt' for\n",
    "the console messages, e.g. `bernoulli-201912081451-1.csv`. Output files\n",
    "written to the temporary directory contain an additional 8-character\n",
    "random string, e.g. `bernoulli-201912081451-1-5nm6as7u.csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The sampler output files are written to a temporary directory which\n",
    "is deleted upon session exit unless the ``output_dir`` argument is specified.\n",
    "The ``save_csvfiles`` function moves the CmdStan CSV output files\n",
    "to a specified directory without having to re-run the sampler.\n",
    "The console output files are not saved. These files are treated as ephemeral; if the sample is valid, all relevant information is recorded in the CSV files."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sampler Progress\n",
    "\n",
    "Your model make take a long time to fit.  The `sample` method provides two arguments:\n",
    "    \n",
    "- visual progress bar:  `show_progress=True`\n",
    "- stream CmdStan ouput to the console - `show_console=True`\n",
    "    \n",
    "To illustrate how progress bars work, we will run the bernoulli model. Since the progress bars are only visible while the sampler is running and the bernoulli model takes no time at all to fit, we run this model for 200K iterations, in order to see the progress bars in action."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fit = model.sample(data=data_file, iter_warmup=100000, iter_sampling=100000, show_progress=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Stan language `print` statement can be use to monitor the Stan program state.\n",
    "In order to see this information as the sampler is running, use the `show_console=True` argument.\n",
    "This will stream all CmdStan messages to the terminal while the sampler is running.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fit = model.sample(data=data_file, chains=2, parallel_chains=1, show_console=True)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running a data-generating model\n",
    "\n",
    "In this example we use the CmdStan example model\n",
    "[data_filegen.stan](https://github.com/stan-dev/cmdstanpy/blob/master/docs/notebooks/data_filegen.stan)\n",
    "to generate a simulated dataset given fixed data values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_datagen = CmdStanModel(stan_file='bernoulli_datagen.stan')\n",
    "datagen_data = {'N':300, 'theta':0.3}\n",
    "fit_sim = model_datagen.sample(data=datagen_data, fixed_param=True)\n",
    "fit_sim.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compute, plot histogram of total successes for `N` Bernoulli trials with chance of success `theta`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "drawset_pd = fit_sim.draws_pd()\n",
    "drawset_pd.columns\n",
    "\n",
    "# restrict to columns over new outcomes of N Bernoulli trials\n",
    "y_sims = drawset_pd.drop(columns=['lp__', 'accept_stat__'])\n",
    "\n",
    "# plot total number of successes per draw\n",
    "y_sums = y_sims.sum(axis=1)\n",
    "y_sums.astype('int32').plot.hist(range(0,datagen_data['N']+1))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
